Papers with machine translation task
AligNART: Non-autoregressive Neural Machine Translation by Jointly Learning to Estimate Alignment and Translate (2021.emnlp-main)
Copied to clipboard
| Challenge: | Non-autoregressive neural machine translation models suffer from the multi-modality problem . aligNART leverages full alignment information to explicitly reduce the modality of the target distribution . |
| Approach: | They propose an alignment decomposition method which explicitly reduces the modality of the target distribution. |
| Outcome: | The proposed model outperforms previous models that focus on modality reduction on two translation tasks. |
On the Importance of Word Boundaries in Character-level Neural Machine Translation (D19-56)
Copied to clipboard
| Challenge: | Neural Machine Translation models typically use a fixed-size lexical vocabulary . subword segmentation methods rely on statistical heuristics that lack any linguistic notion . |
| Approach: | They propose a hierarchical decoding architecture for character-level NMT using subwords . they propose fewer parameters and a more efficient approach to perform translation at the level of words . |
| Outcome: | The proposed model can reach higher translation accuracy than the subword-level model with fewer parameters while maintaining longer-distance contextual and grammatical dependencies. |
Neural Text Normalization with Subword Units (N19-2)
Copied to clipboard
| Challenge: | Text normalization (TN) is an important step in conversational systems. |
| Approach: | They frame text normalization as a machine translation task and tackle it with sequence-to-sequence models. |
| Outcome: | The proposed model normalizes written text to its spoken form to facilitate speech recognition and text-to-speech synthesis. |
Speeding Up Entmax (2022.findings-naacl)
Copied to clipboard
| Challenge: | Recent studies suggest that sparsity is a problem when the trained model is used for inference. |
| Approach: | They propose an alternative to softmax that produces a dense probability distribution but is slower than softmax. |
| Outcome: | The proposed method keeps its virtuous characteristics but is slower than softmax and achieves on par or better performance in machine translation task. |
Neural Machine Translation for Bilingually Scarce Scenarios: a Deep Multi-Task Learning Approach (N18-1)
Copied to clipboard
| Challenge: | Neural machine translation requires large amount of parallel training text to learn a reasonable quality translation model. |
| Approach: | They propose a multi-task learning approach that leverages monolingual linguistic resources in the source side of a machine translation task. |
| Outcome: | The proposed approach is effective on three translation tasks: English-to-French, English- to-Farsi, and English-à-Vietnamese. |
Target Foresight Based Attention for Neural Machine Translation (N18-1)
Copied to clipboard
| Challenge: | Empirical experiments on Chinese-to-English and Japanese-to English datasets show that the proposed attention model delivers significant improvements in terms of alignment error rate and BLEU. |
| Approach: | They propose to explicitly access the target foresight word in the attention model to improve alignment and translation accuracy. |
| Outcome: | Empirical results show that the proposed model improves alignment error rate and BLEU on Chinese-to-English and Japanese-toEnglish datasets. |
Towards Better Modeling Hierarchical Structure for Self-Attention with Ordered Neurons (D19-1)
Copied to clipboard
| Challenge: | Recent studies have shown that a hybrid of self-attention networks (SANs) and recurrent neural networks (RNNs) outperforms both individual architectures, while not much is known about why the hybrid models work. |
| Approach: | They propose to use an advanced variant of self-attention networks (SANs) to enhance the strength of hybrid models by introducing a syntax-oriented inductive bias to perform tree-like composition. |
| Outcome: | The proposed model outperforms both individual models and a standard hybrid model on a machine translation task. |
Multi-Agent Mutual Learning at Sentence-Level and Token-Level for Neural Machine Translation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) has achieved significant progress over recent years. |
| Approach: | They extend mutual learning to the machine translation task and operate at both the sentence-level and the token-level. |
| Outcome: | The proposed method improves on the IWSLT’14 German-English task and also on the WMT’14 English-German task. |
Controlling Text Complexity in Neural Machine Translation (D19-1)
Copied to clipboard
| Challenge: | Prior work on text complexity has focused on simplifying input text in one language, primarily English. |
| Approach: | They propose a method to align news articles written for different levels of target language proficiency. |
| Outcome: | The proposed model outperforms pipeline approaches that translate and simplify text independently. |
Incomplete Utterance Rewriting as Semantic Segmentation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies focus on the task of incomplete utterance rewriting as a machine translation task. |
| Approach: | They propose a semantic segmentation task which incorporates edit operations into the problem and predicts a word-level edit matrix. |
| Outcome: | The proposed approach outperforms existing baselines on several datasets and is four times faster than the standard approach in inference. |
Large Language Models Share Representations of Latent Grammatical Concepts Across Typologically Diverse Languages (2025.naacl-long)
Copied to clipboard
| Challenge: | In the brains of human bilinguals, syntax processing may occur in similar regions for their first and second language, depending on factors like when the second language was learned and language proficiency. |
| Approach: | They propose to use sparse autoencoders to train Llama-3-8B and Aya-23-8B models to train multilingual models that share morphsyntactic representations of grammatical concepts. |
| Outcome: | The proposed model can predict plural verbs in different languages by activating the same plural feature. |
Knowledge Graph Enhanced Neural Machine Translation via Multi-task Learning on Sub-entity Granularity (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods to integrate knowledge graph (KG) with neural machine translation (NMT) have two problems: knowledge under-utilization and granularity mismatch. |
| Approach: | They propose a multi-task learning method on sub-entity granularity to combine machine translation and knowledge reasoning tasks. |
| Outcome: | The proposed method significantly outperforms baseline models on translation tasks and handling the entities. |
Machine Translation Hallucination Detection for Low and High Resource Languages using Large Language Models (2024.findings-emnlp)
Copied to clipboard
Kenza Benkirane, Laura Gongas, Shahar Pelles, Naomi Fuchs, Joshua Darmon, Pontus Stenetorp, David Adelani, Eduardo Sánchez
| Challenge: | Existing methods for detecting hallucinations in machine translation are limited for low-resource languages. |
| Approach: | They evaluate sentence-level hallucination detection approaches using Large Language Models (LLMs) they find that the choice of model is essential for performance. |
| Outcome: | The proposed models outperform the existing models in HRLs and LRLs on average by 0.16 MCC. |
Sequence Length is a Domain: Length-based Overfitting in Transformer Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Current Transformer-based sequence-to-sequence architectures can suffer from overfitting during training. |
| Approach: | They propose to use Transformer-based sequence-to-sequence architectures to overcome overfitting problems when generating very long sequences. |
| Outcome: | The proposed model performs worse on very long sequences than previous approaches on string editing and translation tasks when faced with sequences of length diverging from the length distribution in training data. |
Towards Multiple References Era – Addressing Data Leakage and Limited Reference Diversity in Machine Translation Evaluation (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent research shows a weak correlation between n-gram-based metrics and human evaluations in machine translation tasks. |
| Approach: | They propose to use multiple references generated by LLMs to improve alignment between automatic metrics and human evaluations. |
| Outcome: | The proposed approach improves the alignment between automatic metrics and human evaluations on the WMT22 benchmark with 4 languages and achieves a maximum accuracy gain of 9.5%. |
Set Generation Networks for End-to-End Knowledge Base Population (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge base population systems require a machine translation task to generate multiple facts, but the fact order is not considered. |
| Approach: | They propose a knowledge base population task that aims to discover facts about entities from texts and expand a KB with these facts. |
| Outcome: | The proposed networks achieve state-of-the-art (SoTA) performance on two benchmark datasets. |
Two Counterexamples to Tokenization and the Noiseless Channel (2024.lrec-main)
Copied to clipboard
| Challenge: | Nevertheless, Rényi efficiency is not perfect and the metric is difficult to evaluate because training multiple tokenizers can be prohibitively expensive and takes days or weeks. |
| Approach: | They propose to use Rényi efficiency as an intrinsic mechanism to evaluate a tokenizer for NLP tasks without the expensive step of training multiple models with different tokenizers. |
| Outcome: | The proposed metric is better correlated to downstream model performance than a percentile frequency metric. |